Use asyncio by rwols · Pull Request #2880 · sublimelsp/LSP

rwols · 2026-04-22T17:34:21Z

This PR switches the codebase to using async def functions and asyncio. The loop provider is sublime_aio.

should be merged (and released) at the same time as:

The main driver for doing this is to decrease the thread usage of this plugin from O(n) to O(1) threads, where n is the number of language servers running. The secondary driver is syntax sugar.

Why is this PR so large? Please read: What color is your function?

Self-contained bits:

Topic	Old Way	New Way
Running a short function on a thread that's not the main thread	`sublime.set_timeout_async`	`LSP.plugin.core.aio.call_soon_threadsafe`
Defining a function that's asynchronous	`def f() -> Promise[T]: ...`	`async def f() -> T: ...`
Chaining asynchronous functions	`Promise.then(lambda x: ...)`	`x = await f()`
Doing something when a server request is done	`session.send_request_async(R(), lambda x: ...)`	`x = await session.request(R())`
Doing something when a server request fails	define an on_error callback	use a `try ... except ResponseException:` block
Handling partial request results	define an on_partial_result callback	~~`async for partial_result in session.stream(R()):` (caveat: only works for `list[...]`-style responses)~~
Starting a coroutine from a regular function	n/a	`LSP.plugin.core.aio.run_coroutine_threadsafe(f())`
Awaiting old-style `Promise` objects	`Promise.then`	`await promise`
Waiting for all asynchronous operations to complete	`Promise.all`	`asyncio.gather`
Doing something later	`sublime.set_timeout_async(f, timeout_ms=1000)`	`await asyncio.sleep(1)`
Enforcing a critical section	use `threading.Lock`, or write very complicated queueing logic	use `asyncio.Lock`
Wrapping a new `async` function in a `Promise`	n/a	`Promise.wrap_task`

`f` calls `g`	async `g`	blocking `g`
async `f`	`async def f(): await g()`	`async def f(): g()`
blocking `f`, guaranteed called from asyncio thread	use `aio.TaskContainer.create_task(g())`	`def f(): g()`
blocking `f`, any thread	`def f(): aio.run_coroutine_threadsafe(g())`, or use `aio.TaskContainer.create_task_threadsafe(g())`	`def f(): g()`

The Plan

Make "most" code run on the sublime_aio thread

Most code is doing bookkeeping. This type of code used to run on the Sublime "async" thread. It should run on the asyncio loop thread.

Previously, the code attempted to make most code run on the ST async thread. We never really enforced this. We tried to make it clear that a function/method should be running on the ST async thread by suffixing it with _async.

If you have an async def coroutine function, then such a coroutine function is forced to run on the asyncio loop thread. So enforcement becomes automatic.

Keep `_async` suffixes, assume they run on the asyncio thread

When a method or function has the suffix _async in its name, we tried to ensure these functions run on the ST async thread. These can now be assumed to be running on the asyncio thread.

Make compute-intensive function run on the Sublime "async" thread

The only compute-intensive code we deal with are parsing and emitting JSON. Only the JSON parser/emitter should run on the ST async thread.

Bridging code for existing LSP-* plugins

We made sure that all AbstractPlugin and LspPlugin related (class)methods ran on the ST async thread. I want to now make sure all these (class)methods run on the sublime_aio thread with this pull request.

Certain methods may also be marked async for LspPlugin, most notably on_pre_start and perhaps on_initialize.

The Promise object can be awaited, so older AbstractPlugin/LspPlugin-related functionality returning promises from request handlers work.

Add a test that checks that "tcp server mode" works. Server meaning that this plugin acts as the TCP server and the langserver connects as TCP client.

Conflicts: tests/server.py

netlify · 2026-04-22T17:34:36Z

✅ Deploy Preview for sublime-lsp ready!

Name	Link
🔨 Latest commit	`ea8cee1`
🔍 Latest deploy log	https://app.netlify.com/projects/sublime-lsp/deploys/6a61dae39934e3000816347f
😎 Deploy Preview	https://deploy-preview-2880--sublime-lsp.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

…se later

Also write the rest of the requests in terms of Session.request

rchl · 2026-04-22T20:42:39Z

Would be useful to split it into smaller chunks, if possible. For example it would likely be possible to make start_async async by making it return a Promise and then later convert it to asyncio easily.

Lots of assumptions on my side but that's what I feel.

I was actually looking into that before as I wanted to move start_async into dedicated thread so that it doesn't black other plugins (kinda opposite goal of yours but also kinda similar as I guess with asyncio it will also run on dedicated thread). See #2863

It will be hard to review it properly with a big dump of code that refactors most of the code base.

- Add sublime.set_timeout executor wrapper - Make all request handlers `async` - Define a CancellableInflightStreamingRequest class that enables `async for` syntax - Start inheriting DocumentSyncListener from sublime_aio.ViewEventListener (This one doesn't work yet) The state is fairly broken at this point.

Co-authored-by: Rafał Chłodnicki <rchl2k@gmail.com>

rchl

I have this reproducible crash that ends up breaking other stuff.

I'm working on python files on which I have basedpyright and ruff running.
I open my test HelloWorld.java file which triggers initialization of LSP-jdtls
For one reason or another jdtls fails to start on my system. LSP prints to the console:

LSP: jdtls crashed (1 / 5 times in the last 180.0 seconds), exit code 13, exception: None

Now on opening new python files basedpyright and ruff don't start on those files anymore.

The example java file itself has this code:

public class HelloWorld {
    // Your program begins with a call to main()
    public static void main(String[] args) {
        // Prints "Hello, World" to the terminal window.
        System.out.println("Hello, World");
    }
}

On main it tries to start 5 times and each time prints more useful error message (server's output):

LSP: jdtls crashed (1 / 5 times in the last 180.0 seconds), exit code 0, exception: Unexpected payload in server's stdout:


An error has occurred. See the log file
/Users/rafal/Library/Caches/Sublime Text/Package Storage/LSP-jdtls/data/.metadata/.log.

LSP: jdtls crashed (2 / 5 times in the last 180.0 seconds), exit code 0, exception: Unexpected payload in server's stdout:


An error has occurred. See the log file
/Users/rafal/Library/Caches/Sublime Text/Package Storage/LSP-jdtls/data/.metadata/.log.

...

rchl · 2026-08-22T13:45:02Z

    """Holds state per request."""

-    def __init__(self, sv: SessionViewProtocol, request_id: int, request: Request[Any, Any]) -> None:
+    def __init__(self, sv: SessionViewProtocol, cancellable: RequestController, request: Request[Any, Any]) -> None:


Let's rename cancellable to controller

rchl · 2026-08-22T13:52:55Z

+
+
+def run_coroutine(
+    coroutine: Coroutine[object, object, T], *, exception_policy: ExceptionPolicy = ExceptionPolicy.STACKTRACE


Seems like exception_policy is never used so why introduce it now?

Even if we think of potential use for it later, I don't think that ExceptionPolicy.MESSAGEBOX would be that useful. More likely someone would want to show some custom error rather than just dump the exception in the message box.

rchl · 2026-08-22T14:00:11Z

+    return _run_on_st_thread(sublime.set_timeout_async, f, *args, **kwargs)
+
+
+def tick(n: int = 1) -> asyncio.Future[None]:


I have hard time thinking of a use case for being able to specify the exact number of ticks. Can we remove that (not currently used) feature?

rchl · 2026-08-22T14:04:05Z

+    flattened list of Exceptions that occurred for each coroutine. BaseExceptions are filtered out.
+    """
+    exceptions: list[Exception] = []
+    items: list[BaseException | list[Exception]] = await asyncio.gather(*coros, return_exceptions=True)


Redundant annotation

Suggested change

items: list[BaseException | list[Exception]] = await asyncio.gather(*coros, return_exceptions=True)

items = await asyncio.gather(*coros, return_exceptions=True)

rchl · 2026-08-22T14:24:59Z


 def show_summary_message(
-    window: sublime.Window, result: ApplyWorkspaceEditResult, summary: WorkspaceEditSummary
+    window: sublime.Window, result: ApplyWorkspaceEditResult, summary: WorkspaceEditSummary | BaseException


Does it still hold that the summary can be a BaseException? At least the types at call sites suggest that it can't.

rchl · 2026-08-22T14:30:50Z

+    view = await open_file(window, decoded_uri, flags, group)
    if view:
-        return center_selection(view, r)
-    return None
+        if fragment := urlparse(decoded_uri).fragment:
+            if selection := lsp_range_from_uri_fragment(fragment):
+                center_selection(view, selection)
+    return view


less lines

if view and (fragment := urlparse(decoded_uri).fragment) and (selection := lsp_range_from_uri_fragment(fragment)): center_selection(view, selection)

rchl · 2026-08-22T14:37:08Z

+            def on_main_thread() -> None:
+
+                # window.open_file brings the file to focus if it's already opened, which we don't want (unless it's
+                # supposed to open as a separate view).
+                view = _find_open_file(window, file)
+                if view and _return_existing_view(flags, window.get_view_index(view)[0], window.active_group(), group):
+                    loop.call_soon_threadsafe(lambda: resolve_right_now(view))
+                    return
+
+                was_already_open = view is not None
+                if not was_already_open and not os.path.isfile(file):
+                    # window.open_file creates a new view with empty content if the path from the given URI doesn't
+                    # exist as a file on disk, but we don't want that here. If the language server wants to create a new
+                    # file for a given URI, it must use the CreateFile resource operation in a WorkspaceEdit.
+                    loop.call_soon_threadsafe(lambda: resolve_right_now(view))
+                    return
+
+                view = window.open_file(file, flags, group)
+                if not view.is_loading():
+                    if was_already_open and (flags & sublime.NewFileFlags.SEMI_TRANSIENT):
+                        # workaround bug https://github.com/sublimehq/sublime_text/issues/2411 where transient view
+                        # might not get its view listeners initialized.
+                        sublime_plugin.check_view_event_listeners(view)  # type: ignore
+                    # It's already loaded. Possibly already open in a tab.
+                    loop.call_soon_threadsafe(lambda: resolve_right_now(view))
+
+                loop.call_soon_threadsafe(resolve_later)


Our convention is not to have blank lines in function bodies:

Suggested change

def on_main_thread() -> None:

# window.open_file brings the file to focus if it's already opened, which we don't want (unless it's

# supposed to open as a separate view).

view = _find_open_file(window, file)

if view and _return_existing_view(flags, window.get_view_index(view)[0], window.active_group(), group):

loop.call_soon_threadsafe(lambda: resolve_right_now(view))

return

was_already_open = view is not None

if not was_already_open and not os.path.isfile(file):

# window.open_file creates a new view with empty content if the path from the given URI doesn't

# exist as a file on disk, but we don't want that here. If the language server wants to create a new

# file for a given URI, it must use the CreateFile resource operation in a WorkspaceEdit.

loop.call_soon_threadsafe(lambda: resolve_right_now(view))

return

view = window.open_file(file, flags, group)

if not view.is_loading():

if was_already_open and (flags & sublime.NewFileFlags.SEMI_TRANSIENT):

# workaround bug https://github.com/sublimehq/sublime_text/issues/2411 where transient view

# might not get its view listeners initialized.

sublime_plugin.check_view_event_listeners(view) # type: ignore

# It's already loaded. Possibly already open in a tab.

loop.call_soon_threadsafe(lambda: resolve_right_now(view))

loop.call_soon_threadsafe(resolve_later)

def on_main_thread() -> None:

# window.open_file brings the file to focus if it's already opened, which we don't want (unless it's

# supposed to open as a separate view).

view = _find_open_file(window, file)

if view and _return_existing_view(flags, window.get_view_index(view)[0], window.active_group(), group):

loop.call_soon_threadsafe(lambda: resolve_right_now(view))

return

was_already_open = view is not None

if not was_already_open and not os.path.isfile(file):

# window.open_file creates a new view with empty content if the path from the given URI doesn't

# exist as a file on disk, but we don't want that here. If the language server wants to create a new

# file for a given URI, it must use the CreateFile resource operation in a WorkspaceEdit.

loop.call_soon_threadsafe(lambda: resolve_right_now(view))

return

view = window.open_file(file, flags, group)

if not view.is_loading():

if was_already_open and (flags & sublime.NewFileFlags.SEMI_TRANSIENT):

# workaround bug https://github.com/sublimehq/sublime_text/issues/2411 where transient view

# might not get its view listeners initialized.

sublime_plugin.check_view_event_listeners(view) # type: ignore

# It's already loaded. Possibly already open in a tab.

loop.call_soon_threadsafe(lambda: resolve_right_now(view))

loop.call_soon_threadsafe(resolve_later)

rchl · 2026-08-22T14:38:34Z

+                    # It's already loaded. Possibly already open in a tab.
+                    loop.call_soon_threadsafe(lambda: resolve_right_now(view))
+
+                loop.call_soon_threadsafe(resolve_later)


Missing return here, I think.

rchl · 2026-08-22T14:47:24Z

+    @deprecated("use SessionBuffer.request_code_actions instead")
    def request_code_actions_async(


Is this actually used anywhere? I can't find any references. Not even in packages.

rchl · 2026-08-22T14:49:07Z

+    @deprecated("use Session.run_code_action instead")
+    def run_code_action_async(


I think this can be removed.

rwols added 13 commits April 13, 2026 23:50

Test both TCP modes

699dad3

Add a test that checks that "tcp server mode" works. Server meaning that this plugin acts as the TCP server and the langserver connects as TCP client.

Use enum for 'Mode' in server.py

908c849

Make init_options optional

b4f0420

WIP add asyncio to transports

76f3c3f

Test both TCP modes

45d7ea6

Add a test that checks that "tcp server mode" works. Server meaning that this plugin acts as the TCP server and the langserver connects as TCP client.

Use enum for 'Mode' in server.py

e8fae30

Make init_options optional

c511cec

Use StrEnum

7438d4a

Merge branch 'chore/add-tests' into feat/asyncio

6a104b0

More work on adding asyncio

d59bb1c

Merge branch 'main' into feat/asyncio

c0bea09

Conflicts: tests/server.py

More work towards asyncio

c1fc59c

Remove accidental merge conflict changes

e35b611

predragnikolic reviewed Apr 22, 2026

View reviewed changes

Comment thread tests/server.py Outdated

This comment was marked as resolved.

Sign in to view

rwols added 3 commits April 22, 2026 20:17

Remove code in server.py, I really screwed up the merge but will reba…

6cad098

…se later

refactor Promise.__await__

1597782

Add more comments to _SetTimeoutAsyncExecutor

cc642f4

This comment was marked as outdated.

Sign in to view

Rename Session._invoke_views -> Session._invoke_views_async

cea0e99

Also write the rest of the requests in terms of Session.request

rchl reviewed Apr 22, 2026

View reviewed changes

Comment thread plugin/core/promise.py Outdated

jwortmann reviewed Apr 22, 2026

View reviewed changes

Comment thread plugin/core/windows.py Outdated

rchl reviewed Apr 22, 2026

View reviewed changes

Comment thread plugin/core/windows.py Outdated

rwols added 5 commits April 28, 2026 18:53

debugging sublime_aio.ViewEventListener

472705b

Merge branch 'main' into feat/asyncio

ea34714

Fix calls

069ed50

More fix calls

776749a

rwols force-pushed the feat/asyncio branch from e976681 to f96085a Compare July 22, 2026 10:46

Merge branch 'main' into feat/asyncio

42f94d3

rwols force-pushed the feat/asyncio branch from ea8cee1 to 42f94d3 Compare July 23, 2026 09:14